Inequalities for Shannon Entropy and Kolmogorov Complexity
نویسندگان
چکیده
منابع مشابه
Inequalities for Shannon Entropy and Kolmogorov Complexity
It was mentioned by Kolmogorov (1968, IEEE Trans. Inform. Theory 14, 662 664) that the properties of algorithmic complexity and Shannon entropy are similar. We investigate one aspect of this similarity. Namely, we are interested in linear inequalities that are valid for Shannon entropy and for Kolmogorov complexity. It turns out that (1) all linear inequalities that are valid for Kolmogorov com...
متن کاملShannon Entropy vs. Kolmogorov Complexity
Most assertions involving Shannon entropy have their Kolmogorov complexity counterparts. A general theorem of Romashchenko [4] states that every information inequality that is valid in Shannon’s theory is also valid in Kolmogorov’s theory, and vice verse. In this paper we prove that this is no longer true for ∀∃-assertions, exhibiting the first example where the formal analogy between Shannon e...
متن کاملSome equivalences between Shannon entropy and Kolmogorov complexity
[24] A. N. Kolmogorov, “On the approximation of distributions of 1271 G. H. Hardy, J. E. Litt lewood, and G. Polya, ZnequaIit ies. New sums of independent summa nds by infinitely divisible distributions,” San/&v& vol. 25, pp. 159-174, 1963. [28] York and London: Cambridge Univ. Press? 1934. D. E. Da kin and C. J. Eliezer, “General ization of Holder’s and (251 A. Renyi, “On the amount of missing...
متن کاملConceptualizing Birkhoff's Aesthetic Measure Using Shannon Entropy and Kolmogorov Complexity
In 1928, George D. Birkhoff introduced the Aesthetic Measure, defined as the ratio between order and complexity, and, in 1965, Max Bense analyzed Birkhoff’s measure from an information theory point of view. In this paper, the concepts of order and complexity in an image (in our case, a painting) are analyzed in the light of Shannon entropy and Kolmogorov complexity. We also present a new vision...
متن کاملInequalities for Shannon entropies and Kolmogorov complexities
The paper investigates connections between linear inequalities that are valid for Shannon entropies and for Kolmogorov complexities.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of Computer and System Sciences
سال: 2000
ISSN: 0022-0000
DOI: 10.1006/jcss.1999.1677